Visualizing Outliers

Outliers have more than two centuries’ history in the field of statistics. Recently, they have become a focal topic because of their relevance to terrorism, network intrusions, financial fraud, and other areas where rare events are critical to understanding a process. This paper presents a new algorithm, called hdoutliers, for detecting multidimensional outliers. It is unique for a) dealing with a mixture of categorical and continuous variables, b) dealing with the curse of dimensionality (many columns of data), c) dealing with many rows of data, d) dealing with outliers that mask other outliers, and e) dealing consistently with unidimensional and multidimensional datasets. Unlike ad hoc methods found in many machine learning papers, hdoutliers is based on a distributional model that allows outliers to be tagged with a probability.